Method and System of Voice Carry Over for Instant Messaging Relay Services

ABSTRACT

A method of assisting communication for a user is provided. The method includes receiving an IM message including a request for a voice carry over from the user, and transmitting to the user an invitation to join a first voice connection. The method further includes initiating the first voice connection with the user, and initiating a second voice connection with a recipient. Additionally, the method includes communicating to the recipient a first voice communication from the user over the first and second voice connections, and communicating to the user a response IM message including a transcribed version of a second voice communication from the recipient. An apparatus for assisting communication for a user is provided. A computer-readable medium having stored thereon computer-executable instructions is provided. The computer-executable instructions cause a processor to perform a method when executed.

This application is a continuation of U.S. patent application Ser. No.13/742,629, filed Jan. 16, 2013, which is a continuation of U.S. patentapplication Ser. No. 12/627,397, filed on Nov. 30, 2009, now U.S. Pat.No. 8,385,329, the disclosures of which are incorporated herein byreference in their entirety.

FIELD OF THE INVENTION

The system and method of the present application relates tocommunication services for deaf or hearing-impaired persons, and inparticular relates to a voice carry over (VCO) relay and a hearing carryover (HCO) relay. In particular, the system and method provided relatesto an instant messaging relay system in including VCO and HCO.

BACKGROUND OF THE INVENTION

Relay services provide deaf and hearing impaired people the ability tomake phone calls. A deaf customer types what they wish to say to theperson they are calling (also referred to as the recipient or the farparty), and the relay communications assistant (CA) voices the text tothe hearing party being called. When the hearing party speaks, the CAtypes what they say for transmission to a screen for the deaf caller toread.

Many deaf or hearing impaired users can speak and want the ability totalk instead of type for their outgoing part of the conversation. Theonly part of relay they require or desire is the translation of theother party's voice into text that they can read. This ‘one-way’ relayis known as voice carry over (VCO).

SUMMARY OF THE INVENTION

Instant messaging relay (IM Relay) allows users to communicate throughan instant messaging system, for instance AOL Instant Messenger (AIM),to hearing users. An IM system may allow voice transport. The AIM usermay simply press (ALT+T) to turn on voice communication. IM Relay allowsa deaf or hearing impaired user to type instant messages (IM messages)to a CA and have the CA speak the typed communications to a recipient.The recipient then speaks to the CA, who transcribes the words of therecipient and sends the transcription in an IM message to the deaf orhearing impaired person. Conventional IM Relay, however, does notsupport VCO calls. As a new feature, IM Relay could route this voicetransmission to the hearing customer, thus creating IM VCO.

A method of assisting communication for a user is provided. The methodincludes receiving an IM message including a request for a voice carryover from the user, and transmitting to the user an invitation to join afirst voice connection. The method further includes initiating the firstvoice connection with the user, and initiating a second voice connectionwith a recipient. Additionally, the method includes communicating to therecipient a first voice communication from the user over the first andsecond voice connections, and communicating to the user a response IMmessage including a transcribed version of a second voice communicationfrom the recipient.

These and other advantages of the invention will be apparent to those ofordinary skill in the art by reference to the following detaileddescription and the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a system according to an exemplary embodiment;

FIG. 2 illustrates a screen shot for an exemplary embodiment;

FIG. 3 illustrates schematically a system according to an exemplaryembodiment;

FIG. 4 illustrates schematically a system according to another exemplaryembodiment;

FIG. 5 illustrates a method according to an exemplary embodiment; and

FIG. 6 is a block diagram of a computer in accordance with an exemplaryembodiment.

DETAIL DESCRIPTION

FIG. 1 illustrates a system according to an exemplary embodiment. InFIG. 1, solid lines may represent voice or text communications, anddashed lines represent text communications. System 100 operates toprovide communication services for deaf user 110. Deaf user 110 may becompletely deaf, or alternatively, hearing impaired.

Deaf user 110 may desire to initiate a telephone call with hearing user150. Deaf user 110 may use a personal computer or other electronicdevice to send an instant message to screen name 120, for instance“ATTVC01”, which may represent the screen name of a communicationassistant able to conduct voice carry over calls. The instant messagemay be sent to initiate the voice carry over call from deaf user 110 tohearing user 150. The instant message sent to screen name 120 may berouted to, or activate, VCO module 130. VCO module 130 may be astand-alone computer, a mini laptop, or another processor. VCO module130 may interact with communication assistant (CA) 140. CA 140 may be apersonal computer (PC) or another terminal, or may represent a personoperating such a PC or terminal.

CA 140 opens a voice link with deaf user 110 by sending an invitation todeaf user 110. CA 140 may interact with CAP/SNAP 160, which may be amaster computer for the relay services system. CAP/SNAP 160 maycommunicate with IM gateway 170, which may send an instant message fromscreen name 180, for instance “ATTRelay” to deaf user 110. Screen name180 may represent a screen name of the communication assistanceoperating a text portion of the relay. After deaf user 110 responds inthe affirmative to the instant message invitation for the voicecomponent of the voice carry over, an audio link may be formed betweenCA 140 and deaf user 110. CA 140 may then complete a voice call tohearing user 150. The voice call from CA 140 to hearing user 150 may bea two way voice communication on a public telephone network (forinstance a public switched telephone network (PSTN)), or alternativelymay be a VoIP phone call on the internet. During the phone call, deafuser 110 may communicate by voice through VCO module 130 and CA 140 tohearing user 150, and hearing user 150 may communicate by voice with CA140. CA 140 may transcribe the voice communications of hearing user 150and send the transcribed voice communications via CAP/SNAP 160 and IMgateway 170 to deaf user 110 from screen name 180.

FIG. 2 illustrates a screen shot of an exemplary system. Screen shot 200is for a screen name “Relay user1” representing a user. Screen shot 200identifies two other screen names with which the user may communicate byIM, namely “attrelay” 205 and “AttVCO1” 210. Both “attrelay” 205 and“AttVCO1” 210 may represent a communication assistant able to assist theuser with a relay, voice carryover, hearing carryover, or captionedcall. The user presented with screen shot 200 may have the option toselect voice carry over initiation button 220. Additionally, initiationbutton 220 may include hypertag 230 which pops up when a cursor or otherselector passes over, or hovers over, initiation button 220. Afteractivating VCO initiation button 220, a VCO relay phone call may beconducted.

FIG. 3 illustrates schematically a voice carry over (VCO) system 300according to an exemplary embodiment. Deaf user 110 may be able to speakand prefer to communicate by voice directly to hearing user 150.Therefore, deaf user 110 may communicate speech 305 that is received ata microphone to become voice communication 310. Voice communication 310may be transmitted to hearing user 150 via CA 140. When hearing user 150communicates to deaf user 110, hearing user 150 speaks creating voicecommunication 320. Voice communication 320 is heard by CA 140, whotranscribes the content of voice communication 320. CA 140 then sendstext 330, comprising the transcribed content of voice communication 320,by IM message to deaf user 110.

FIG. 4 illustrates an alternative embodiment including a hearing carryover (HCO) system 400 (also referred to as a caption system). HCO system400 may be helpful for a hearing impaired person who wants to hear theperson with whom they are speaking, but who also requires atranscription of the voice communication. Deaf user 110 (who may, asdiscussed above, be merely hearing impaired) may communicate speech 305creating voice communication 310, which is transmitted to hearing user150. Alternatively, voice communication 310 may be a text communicationin the case of a user who does not want to, or is unable to, speak.Hearing user 150 may communicate by voice communication 320 to CA 140,in the same manner as FIG. 3. Additionally, in the same manner as FIG.3, CA 140 may transcribe voice communication 320 to create textcommunication 330 for communication to deaf user 110 by an instantmessaging system. However, in addition, voice communication 410 may betransmitted by CA 140 to deaf user 110. Voice communication 410 may beidentical to voice communication 320, and may merely be a continuationof voice communication 320.

The system discussed above in regard to FIG. 4, in which deaf user 110communicates by text to hearing user 150, and voice communication 410,but not text communication 330, is provided to deaf user 110, isconventionally described as hearing carry over. The system discussedabove in which deaf user 110 communicates by voice communication 310 tohearing user 150, and voice communication 410 and text communication 330are provided to deaf user 110, is conventionally described ascaptioning.

FIG. 5 illustrates a method according to an exemplary embodiment. Theflow of the method shown in FIG. 5 starts in start circle 500. The flowproceeds from start circle 500 to operation 510, which receives an IMmessage including a request for a voice carry over from the user. Theuser may be deaf or hearing impaired, and the request may be directed toa communication assistant (CA). From operation 510, the flow proceeds tooperation 520, which transmits to the user an invitation to join a voicecommunication. An additional step in the method may provide foraccepting by the user of the invitation from the communication assistantto join the voice communication. From operation 520, the flow proceedsto operation 530, which initiates a first voice communication with theuser. From operation 530, the flow proceeds to operation 540, whichinitiates a second voice communication with a recipient. From operation540, the flow proceeds to operation 550, which communicates to therecipient a first voice communication from the user. From operation 550,the flow proceeds to operation 560, which communicates to the user aresponse IM message including a transcribed version of a second voicecommunication from the recipient. From operation 560, the flow proceedsto end circle 570.

An exemplary method may further include accepting by a user theinvitation to join the first voice connection. The initiating of thefirst voice connection with the user may be performed by a communicationassistant, and the initiating of the second voice connection with therecipient may be performed by the communication assistant.

The IM message may further include a request for a hearing carry over tothe user, and the method may further include communicating to the userthe second voice communication from the recipient.

The first voice connection with the user may be over an IM voicefunctionality or over a VoIP connection. The second voice connectionwith the recipient may be over a telephone connection or over a VoIPconnection.

The method may further include accepting an initiation of an IMcommunication from the user. The initiation of the IM communication maybe the same message as the initial IM message from the deaf or hearingimpaired user to the communication assistant, or may precede thereceiving of the IM message.

FIG. 6 is a high level block diagram of a computer in accordance with anembodiment of the present invention. Computer 600 can, for example,operate as VCO module 130, a terminal operated by CA 140, CAP/SNAP 160,or any other terminal. Additionally, computer 600 can perform the stepsdescribed above (e.g., with respect to FIG. 5). Computer 600 containsprocessor 610 which controls the operation of computer 600 by executingcomputer program instructions which define such operation, and which maybe stored on a computer-readable recording medium. The computer programinstructions may be stored in storage 620 (e.g., a magnetic disk, adatabase) and loaded into memory 630 when execution of the computerprogram instructions is desired. Thus, the computer operation will bedefined by computer program instructions stored in memory 630 and/orstorage 620 and computer 600 will be controlled by processor 610executing the computer program instructions. Computer 600 also includesone or more network interfaces 640 for communicating with other devices,for example other computers, servers, or websites. Network interface 640may, for example, be a local network, a wireless network, an intranet,or the Internet. Computer 600 also includes input/output 650, whichrepresents devices which allow for user interaction with the computer600 (e.g., display, keyboard, mouse, speakers, buttons, webcams, etc.).One skilled in the art will recognize that an implementation of anactual computer will contain other components as well, and that FIG. 6is a high level representation of some of the components of such acomputer for illustrative purposes.

The foregoing Detailed Description is to be understood as being in everyrespect illustrative and exemplary, but not restrictive, and the scopeof the invention disclosed herein is not to be determined from theDetailed Description, but rather from the claims as interpretedaccording to the full breadth permitted by the patent laws. It is to beunderstood that the embodiments shown and described herein are onlyillustrative of the principles of the present invention and that variousmodifications may be implemented by those skilled in the art withoutdeparting from the scope and spirit of the invention.

We claim:
 1. A method comprising: receiving an initial messagecomprising a request for hearing carry over; initiating a first voiceconnection with a user and a second voice connection with a recipient;communicating to the recipient a first voice message from the user;communicating to the user a second voice communication from therecipient over the first voice connection and the second voiceconnection; and communicating to the user an instant message including atranscribed version of the second voice communication.
 2. The method asrecited in claim 1, wherein the initial message comprises an initialinstant message.
 3. The method as recited in claim 1, wherein the firstvoice connection with the user is over an instant message voicefunctionality selectable in an instant message client application. 4.The method as recited in claim 1, wherein the first voice connectionwith the user is over a Voice over Internet Protocol connection.
 5. Themethod as recited in claim 1, wherein the second voice connection withthe recipient is over a telephone connection.
 6. The method as recitedin claim 1, wherein the second voice connection with the recipient isover a Voice over Internet Protocol connection.
 7. The method as recitedin claim 1, further comprising: transmitting to the user an invitationto join the first voice connection.
 8. The method as recited in claim 7,further comprising: receiving an acceptance from the user of theinvitation to join the first voice connection.
 9. An apparatuscomprising: a processor; and a memory to store computer programinstructions, the computer program instructions when executed on theprocessor cause the processor to perform operations comprising:receiving an initial message comprising a request for hearing carryover; initiating a first voice connection with a user and a second voiceconnection with a recipient; communicating to the recipient a firstvoice message from the user; communicating to the user a second voicecommunication from the recipient over the first voice connection and thesecond voice connection; and communicating to the user an instantmessage including a transcribed version of the second voicecommunication.
 10. The apparatus as recited in claim 9, wherein theinitial message comprises an initial instant message.
 11. The apparatusas recited in claim 9, wherein the first voice connection with the useris over an instant message voice functionality selectable in an instantmessage client application.
 12. The apparatus as recited in claim 9,wherein the first voice connection with the user is over a Voice overInternet Protocol connection.
 13. The apparatus as recited in claim 9,wherein the second voice connection with the recipient is over atelephone connection.
 14. The apparatus as recited in claim 9, whereinthe second voice connection with the recipient is over a Voice overInternet Protocol connection.
 15. The apparatus as recited in claim 9,the operations further comprising: transmitting to the user aninvitation to join the first voice connection.
 16. The apparatus asrecited in claim 15, the operations further comprising: receiving anacceptance from the user of the invitation to join the first voiceconnection.
 17. A computer readable medium storing computer programinstructions, which, when executed on a processor, cause the processorto perform operations comprising: receiving an initial messagecomprising a request for hearing carry over; initiating a first voiceconnection with a user and a second voice connection with a recipient;communicating to the recipient a first voice message from the user;communicating to the user a second voice communication from therecipient over the first voice connection and the second voiceconnection; and communicating to the user an instant message including atranscribed version of the second voice communication.
 18. The computerreadable medium as recited in claim 17, wherein the initial messagecomprises an initial instant message.
 19. The computer readable mediumas recited in claim 17, wherein the first voice connection with the useris over an instant message voice functionality selectable in an instantmessage client application.
 20. The computer readable medium as recitedin claim 17, wherein the first voice connection with the user is over aVoice over Internet Protocol connection.